The present invention relates generally to a data processing apparatus and method, a recording medium, and a program and, more particularly, to a data processing apparatus and method, a recording medium, and a program which can easily and infallibly store and recall complicated time-series data.
The applicant hereof disclosed in Japanese Patent Laid-open No. Hei 11-126198 a technology of generating time-series data by use of a neural network of recurrent type.
In the disclosed technology, as shown in FIG. 1, the apparatus is basically configured with a lower-layer network having recurrent neural networks (RNNs) 1-1 through 1-n and a higher-layer network having recurrent neural networks RNNs 11-1 through 11-n.
In the lower-layer network, the outputs of the RNNs 1-1 through 1-n are supplied to a combining circuit 3 via respective gates 12-1 through 12-n.
In the higher-layer network, the outputs of the RNNs 11-1 through 11-n are supplied to a combining circuit 13 via respective gates 12-1 through 12-n. In accordance with the a combined output from the combining circuit 13 of the higher-layer network, the on/off operations of gates 2-1 through 2-n of the lower-layer network are controlled.
The RNNs 1-1 through 1-n of the lower-layer network generate patterns P1 through Pn respectively. On the basis of the output of the combining circuit 13 of the higher-layer network, predetermined one of the gates 2-1 through 2-n of the lower-layer network is turned on/off, thereby causing the combining circuit 3 to selectively output one of the patterns P1 through Pn generated by the predetermined one of the RNNs 1-1 through 1-n.
Consequently, as shown in FIG. 2 for example, patterns which change with time can be generated by generating pattern P1 for a predetermined period and then pattern P2 for another predetermined period and then pattern P1 again for still another predetermined period, for example.
However, in the above-mentioned disclosed technology, the gates 2-1 through 2-n executes a so-called winner-take-all operation, so that it is difficult to store and generate complicated patterns.
It is therefore an object of the present invention to provide a data processing apparatus and method, a recording medium, and a program which are capable of easily and infallibly store and generate patterns even though they are complicated.
In carrying out the invention and according to a first aspect thereof, there is provided a data processing apparatus including: processing means including a first recurrent neural network for applying a predetermined nonlinear function to data inputted from an input terminal, the processing means receiving first data constituted by time-series data and second data constituted by time-series data at the input terminal of the first recurrent neural network to execute the processing; generating means including a second recurrent neural network for generating the second data by applying a predetermined nonlinear function to data inputted from an input terminal; and computing means for executing computation on the second data and third data generated by error back propagation by the first recurrent neural network to generate fourth data.
Preferably, in the above-mentioned data processing apparatus, the generating means generates the second data which change with a longer period than that of the first data.
Preferably, in the above-mentioned data processing apparatus, the computing means executes computation by use of data generated by error back propagation by the first recurrent neural network at the time of learning.
Preferably, in the data processing apparatus, the computing means executes the computation by use of a sigmoid function.
Preferably, in the data processing apparatus, the computing means executes, at the time of learning, a computation including a first computation using data generated by error back propagation by the first recurrent neural network and a second computation for smoothing in an adjacent space-time.
Preferably, in the data processing apparatus, the computing means executes, at the time of future prediction, a computation including a first computation of the second data and a second computation for smoothing in an adjacent space-time.
Moreover, this computing means may execute, at the time of recalling the past, a computation including a first computation of the second data, a second computation using data generated by error back propagation by the first recurrent neural network, and a third computation for smoothing in an adjacent space-time.
In carrying out the invention and according to a second aspect thereof, there is provided a data processing method including: a processing step for performing processing by using a first recurrent neural network for applying a predetermined nonlinear function to data inputted from an input terminal, the processing step receiving, at the input terminal of the first recurrent neural network, first data constituted by time-series data and second data constituted by time-series data to execute the processing of the first and second data; a generating step for performing processing by using a second recurrent neural network for generating the second data by applying a predetermined nonlinear function to data inputted from an input terminal; and a computing step for executing computation on the second data and third data generated by error back propagation by the first recurrent neural network to generate fourth data.
In carrying out the invention and according to a third aspect thereof, there is provided a recording medium recording a computer-readable program, including: a processing step for performing processing by using a first recurrent neural network for applying a predetermined nonlinear function to data inputted from an input terminal, the processing step receiving, at the input terminal of the first recurrent neural network, first data constituted by time-series data and second data constituted by time-series data to execute the processing of the first and second data; a generating step for performing processing by using a second recurrent neural network for generating the second data by applying a predetermined nonlinear function to data inputted from an input terminal; and a computing step for executing computation on the second data and third data generated by error back propagation by the first recurrent neural network to generate fourth data.
In carrying out the invention and according to a fourth aspect thereof, there is provided a program for causing a computer to execute: a processing step for performing processing by using a first recurrent neural network for applying a predetermined nonlinear function to data inputted from an input terminal, the processing step receiving, at the input terminal of the first recurrent neural network, first data constituted by time-series data and second data constituted by time-series data to execute the processing of the first and second data; a generating step for performing processing by using a second recurrent neural network for generating the second data by applying a predetermined nonlinear function to data inputted from an input terminal; and a computing step for executing computation on the second data and third data generated by error back propagation by the first recurrent neural network to generate fourth data.
In the data processing apparatus and method and program associated with the present invention, the second data generated by the second recurrent neural network is supplied to the input terminal of the first recurrent neural network to be processed together with the first data.